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was not found to be a factor in overall performance, and there was no 
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are indications from the study that ability plays a major role in 
assessing the effectiveness of a tutorial versus hypertext 
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high-ability subjects. Six tables and two figures illustrate study 
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An Assessment of the Effectiveness of a Hypertext 
Instructional Delivery System when Compared to a 
Traditional CAI Tutorial 



Ann M. Quade 
Department of Computer and Information Sciences 
Mankato State University 
Mankato, MN 



More than 40 years ago, Vannevar Bush, science advisor to President Roosevelt, 
proposed a hypothetical machine, the MEMIX, that would allow individuals to browse 
through information, photographs, maps, and sketches by using links connecting any two 
source documents. The visionary image provided by Bush coupled with significant 
advances in computers, software, and high capacity fast access digital storage devices 
have given rise to databases that today contain documents, graphics, sound, speech, 
and animated sequences. A database containing information in these varied formats 
which supports the non-sequential access of information by the user has been coined 
hypertext. 

Hypertext can be visualized as a 'semantic' network of nodes and links. In this 
model, nodes represent information units (e.g. documents, graphics, sound, etc.) while 
iinks serve as the cross-reference threads connecting related nodes. The links 
connecting nodes are usually denoted by highlighted words, phrases or graphics and are 
activated by using a mouse or anew keys to select it. Depending on its purpose and 
location, a link can: transfer the reader to a new but related topic, show a reference or 
sen/0 to move from a reference to the article, provide ancillary information, display a 
related illustration, schematic, photograph or video sequence, display an index or access 
and run a related application program. 

The deveic pment of hypertext systems has grown rapidly during the last decade. 
Today, application. j which utilize hypertext include: online help and documentation 
resources, softwara engineering tools, encyclopedias, reference manuals and books, 
and computer assisted learning in numerous educational disciplines. 

Hypertext has three characteristics that are not present in traditional insiaiction 
which relies heavily on a highly directed flow of information. First, hypertext systems allow 
huge collections of information in a variety of mediums to be stored in an extremely 
compact form and accessed easily and readily. The stored information can be linked both 
explicitly and implicitly. Explicit links can be used by authors to suggest paths through 
infomnation which learners may or may not choose to follow. Implicit links can be included 
which support glossaries, dictionaries, and navigational aids to be used as needed by the 
learner (Marchionini, 1988). 

Second, hypertext is an enabling rather than directive environment. Hypertext 
systems en(X)urage the user to probe information sources. By selecting the order, 
number, and composition of the nodes accessed, one can acquire and process 
information in ways that capitalize on their individual cognitive and experiential framework 
(Schneiderman, 1990). Learning theorists contend user initiated movement also 
increases the number of retrieval pathways available to access encoded information from 
the user's own long term menwry (Jonassen ,1988). Not only does hypertext offer a new 
way to learn course content but also offers a new way of learning how to learn. 

Third, hypertext offers the potential to alter the roles of teachers and learners and 
the interactions between them. The Jlexibility of hypertext enables students to create 
unique paths and interpretations of the paths which later can be shared with teachers or 
other students (Marchionini, 1988). 

Conversely, others argue that hypertext is less than optimal. Learner control, 
aptitude-treatment interaction (ATI), disorientation and cognitive overhead are problems 
often associated with hypertext. 

For some learners, the degree of control afforded by hypertext may negatively 
affect learning. Tennyson and Rothen's (1979) research on learner, adaptive, and 
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program control strategies employed in computer based instruction concluded that 
learner control consistently yielded lower post-test scores than other control strategies 
partially because students terminated instruction too early. Because of the richness of 
the medium, learners may not know what sequence or content area of hypertext are best 
for themselves to reach desired learning objectives. Some suggest that the less familiar 
subjects are with the content of the unit to be mastered, the greater the need for support 
in the form of clearly stated learner objectives. Such objectives provided to the learner 
could serve as a 'checklist' indicating to the learner what must be learned while using the 
hypertext system. Objectives used in this manner provide additional guidance and 
direction as learners utilize hypertext for instruction. 

Other issues related to instructional control appear to be directly related to the 
learner himself, Hannafin (1984) concluded from reviewing relevant research that learner 
control is favored over program control when: (1) learners are older and more mature; (2) 
learners are more capable; (3) and higher order skills rather than factual information are 
being taught. Stated another way, hypertext may not be the best mode of instruction for 
all learners or content areas. 

Aptitude-treatment interaction phenomenon may be also be aggravated by the 
lack of structure in such a learning environment. Research by Clark (1982) indicates low- 
ability students have higher achievement from more structured Instructional methods 
whereas they prefer less structured methods. On the other hand, high-ability students, 
he claims, will generally prefer more instructional support but may actually perform better 
when given leaner, less elaborated material where they must take more responsibility for 
generating encoding strategies. Lower ability students, who need the additional support, 
may decline to seek it. He hypothesized that both groups prefer what they perceive will 
demand the lowest 'mental workload' for themselves. 

Additionally, Steinberg (1977) addressed the issue of ATI. She suggested that 
while high achievers seem capable of using most forms of learner control effectively, low 
achievers seem much less able to make decisions about instructional properties (what, 
how, or how much information) than from those decisions involving variations in 
presentational aspects (how information is formatted or delivered). Her research findings 
suggest when instmctional decisions need to be made, better learning is likely to occur 
with external coaching or advisement regarding which resources to select. Aptitude- 
treatment interaction research suggests hypertext may not be the most effective 
environment for low achieving learners. Perhaps a strategy which prescribes closer 
assessment of learning by the instructor may be necessary. 

Disorientation is the result of more degrees of freedom, more dimensions 
available in which one can move. Technical solutions to disorientation are: (1) graphical, 
map-like models which depict the current hyperspace and identify the user's position 
within the space; (2) and providing a means for the learner to backtrack. 

Graphical models termed context webs and local maps are used in Brown 
University's Intermedia system. In this hypertext system, nodes consisting of documents, 
graphics, sound, and animation related to the same context are grouped to form a web. 
Using this rrwdel, every node belongs to one or more web. When the user activates a 
node belonging to a web, they can visually see the other links and nodes which also 
belong to the active web. In addition, a local map can be activated which only shows the 
present active document and its closely related neighbor nodes (Conklin. 1987). 

HyperCard and other developmental shells offer a back-tracking function. By 
clicking on an icon representing a prior screen, the user is able to move directly to that 
screen thus enabling a rapid review of previously encountered information. 

The additional mental effort and concentration required to make choices about 
which paths to follow and which to leave alone can result in 'information myopia' or 
cognitive overhead (Conk'in, 1987). Cognitive overhead may be reduced if the usf . 
able to more effectively organize and integrate the new chunks of information extracted 
from the hypertext medium with existing prior knowledge. 

To date, little research has been conducted to support or resolve the above 
mentioned issues related to hypertext systems. Research performed by Egan, et, al. 
(1987) and Landow (1989) support the contention that hypertext systems do in fact 
support a richer learning environment. Other studies conducted by Covey (1990), Lanza 
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and Roselli (1991), Mays, Kibby, and Watson (1988), and Tripp and Roby (1990) indicate 
hypertext delivery systems were le^-^s effective than other instructional systems. 

This study examines differences in traditional student performance measures 
following a CBI tutorial or a hypertext approach for mastering concepts related to 
intellectual property (copyright and patent) law. 



Method 

Subjects 

Seventy six voluntary subjects in the study were enrolled in an undergraduate 
introductory business law course required by all majors in the College of Business at a 
midwestern state university. The subjects were divided into two treatment groups: one 
group received instruction using a CBI tutorial and the other group received instruction 
delivered in a hypertext format which utilized graphical maps. When queried prior to 
treatment, all participants indicated they had a general awareness of the topic of the 
instnjction, copyright and patent law, but knew nothing about the legal issues and 
ir- 'ications related to the subject area. 

Each subject received an ability ranking by their current business law professor 
prior to the study. The levels were based on eight weeks of observations which included 
assignments and tests. 

Each system also contained a HyperCard script which created an audit trail which 
recorded the movement of each subject as he moved through the the HyperCard stack. 

Programs 

Two computer based applications, a tutorial and hypertext system, were 
developed using HyperCard and its authoring language HyperTalk. Both contained 
identical subject matter but varied in the amount of personal control and stmcture offered 
to the user. 

In the tutorial, students were presented with a main menu listing the topics in the 
proper order of study. Once a topic was selected, the subject was required to move 
through all the related instmction prior to selecting another topic. No restrictions were 
placed on the number of times a topic could be selected for study. 

The hypertext system presented the same topics for study but made no attempt 
to identify or impose an order in which the topics should be studied. In this system, once 
a subject area was selected, the student was presented with a graphical map which 
served to identify the informational nodes associated with the topic as well as depicting 
how they were related. To move to a node shown on the graphical map, the subject was 
required to click on the node. A subject using the hypertext system had three control 
options at any given time: continue to nwve lineariy through the information associated 
with the selected node; return to the graphical map of the topic arKi select another 
related informational node; or return to the list of topics presented and select another 
topic. 

An example of the menus available for both the tutorial and hypertext systems 
(Figures 1 and 2) are appended at the end of the paper. 

Embedded within both systems were numerous opportunities for students to 
check their comprehension of the subject matter through the use of 'what if applications. 
In each situation where responses were solicited and obtained, diagnostic feedback was 
provided. 

Facilities 

Both groups used equipment in an academic computing center at a state 
university in southern Minnesota. The equipment consisted of Macintosh Sf= computer 
systems with internal hard drives. 

Instruments 

The instmment used to measure the students* performance was connposed of 30 
multiple choice questions reflecting both recall and and application of the concepts 
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presented in the software. One point was allocated to each question. This instrument 
was used as both the pre-test and post-test assessment instrument. 

The reliability of this instrument as determined by a KR-20 analysis was .688. 

Procedure 

The procedure used in this study was as follows: 

1 ) Two days prior to the study, subjects unfamiliar with the Macintosh platform used 
to deliver both fomis of instruction received an intensive two hour training 
session. 

2) Treatments were assigned randomly. Detailed instructions on effective use of 
the assigned treatment as well as a practice period using software similar to the 
assigned treatment was implemented for each participant. 

3) After the practice period, subjects were administered a pre-test and given 
objectives reflecting the subject material to be mastered. 

4) Participants were administered a treatment consisting of either the tutorial or 
hypertext system and post-test under supervision of the researcher No time limit 
was placed on viewing the software. 

Results 

Table 1 reports the t-test analysis which was applied to the results of the post-test 
to assess the difference in performance between the tutorial and hypertext treatments. 
No significant difference was found at the .05 level. 



Table 1. Tutorial vs. Hypertext Group Post-Test Results 



^ N Mean s.D. 

Tutorial group 37 20.158 3.6865 

Hypertext group 39 18.313 3.6160 



t(74) df-.45 p..643 



ERIC 



6 

813 



Tutorial vs. Hypertext Instructiorial SystemsS 



A t-test for paired samples (pre-test, post-test) analysis when applied to the two 
treatment groups is represented in Table 2. Significance was found at the ,05 level for 
both treatments. 



Table 2. Tutorial vs. Hypertext 


Group Paired Pre-Test, Post-Test Results 




N 


Mean 


S.D. 


P 


Tutorial group 










Pre-test 


37 


1 J. 4054 


2.544 


.029 


Post-test 


37 


K .5405 


4.018 




Hypertext group 










Pre-test 


39 


13.0513 


3.268 


.005 


Post-test 


39 


18.3205 


3.546 





Table 3 depicts the t-test for paired samples (pre-test, post-test) analysis when 
applied to the three ability groupings. Significance was found at the .05 level for both low 
ability and medium ability groups. 



Table 3. Ability Groups Paired Pre-Test> Post-Test Results 





N 


Mean 


S.D. 


P 


Low ability 










Pre-test 


21 


11.8571 


3.321 


.036 


Post-test 


21 


17.0000 


3.987 




Medium ability 










Pre-test 


36 


12.7500 


2.951 


.030 


Post-test 


36 


18.1389 


3.217 




High ability 










Pre-test 


19 


13.6842 


2.212 


.902 


Post-test 


1 9 


21.5789 


2.893 
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A one-way analysis of variance (Table 4) showed no significance at the .05 level 
between ability and the pre-test scores but significance at the .05 level between ability 
and the post-test scores. Further follow up with the Scheffe procedure indicated ability 
groups one and two (low and medium ability) and groups two and three (medium and high 
ability) werb significantly different at the .05 level. 



Table 4. One Vi/ay Analysis of Variance Pre -Test. Post-test by Ability 



Dependent variable 


Source of Variation 


df 


ss 


MS 


F 


.P . 


Pretest 


Between Groups 


2 


33,3102 


16.6551 


1.962 


.1451 




Within Groups 


73 


616.4267 


8.4031 






Post-tejt 


Between Groups 


2 


229.4839 


114.7420 


10.081 


.0001 




Within Groups 


73 


830.9371 


11.3827 







The data associated with screens viewed per treatment is summarized in Table 5. 
No significant difference between means was found at the .05 level. 



Table 5. Tutorial, Hypertest Screens Viewed 





N 


Mean 


S.D 


Tutorial 


37 


123.824 


26.903 


Hypertext 


■^9 


118.324 


15.813 



A one way analysis of variance (Table 6) showed significance at the .05 level 
between the post-test tutorial scores and ability groupings while no significance was 
found between post-test hypertext scores and ability groups. 



Table 6. One Way Analysis of Variance Treatment Post-test Scores by Ability 



Dependent variable Source of Variation 


df 


SB 


MS 


F 


P 


Tutorial treatment Between groups 


2 


210.6237 


105.3118 


9.5181 


.0005 


Within groups 


34 


376.1871 


11.0643 






Hypertext treatment Between groups 


2 


31.2526 


15.6263 


1.0781 


.3513 


Within groups 


36 


507.3000 


14.4943 







Discussion 

The purpose of this study was to assess, in an undergraduate university setting, 
the effectiveness of a hypertext instructional delivery system when compared to a 
traditional CAI tutorial. Several findings which relate directly to problematic issues of 
learner control and ability were noted. 

Learner control over the number of informational screens viewed was not found 
to be a factor in overall subject performance. Quantitative examination of the audit trail 
compiled by each subject's treatment indicated there was no significant difference in the 
number of subject matter screens viewed by the tutorial versus hypertext treatment 
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groups. Subjects in both treatment groups appear to have taken the treatments 
earnestly. Each treatment group viewed nearly one fourth of subject screens twice. The 
novelty of the instmctional mode should not be mled out in further studies as a factor 
which may play a significant role in this aspect of evaluating learner control. 

Although leamer control in this study did not hinder a subject's ability to navigate 
through the screens, it did not facilitate a greater understanding of the material as 
assessed by the post-test instrument. Neither treatment was found to achieve a greater 
measure of learning of intellectual property law. 

There are indications from this study that ability plays a major role in assessing the 
effectiveness of a tutorial versus hypertext instructional systems. Examination of post- 
test performance when viewed in relation to treatments and ability groupings revealed 
variations. No significant differences in post-test scores were found between the three 
ability groupings in the hypertext treatment. Even though practice using a similar 
hypertext system was done prior to treatment, the subjects' lack of experience and 
unfamiliarity with non-linear learning may have been a factor in these results. Another 
factor may relate to the instrument used for evaluation. Perhaps a different method of 
assessment which emphasizes the broad scope of potential associations made in 
hypterext environments may be more suitable. Significant results were obtained 
between the low and high ability groups as well as the average and high ability groups in 
the tutorial treatment. These results indicated the tutorial was most effective for high 
ability subjects. 

This study was an attempt to assess the effectiveness of tutorial versus hypertext 
delivery systems and to identify learner characteristic which may be used to prescribe the 
most beneficial mode of instruction. Overall, findings indicate learner control was not a 
factor in the hypertext mode of instruction. Additionally, ability appears to play a key role 
in the effectiveness of the tutorial as determined by the post-test instalment while it had 
little if any effect on subject performance as determined by the post-test instrument in the 
hypertext mode. 

This study set forth additional questions to be answered through research. What 
type of instnjment most effectively assesses learning in hypertext environments? Can 
different learning styles or leamer characteristics be used to predict success in a tutorial or 
hypertext system? Does novelty play a role in tutorial, hypertext, or both forms of 
instruction systems? Can quantitative analysis of audit trails provide insight to designers 
of hypertext systems? Answers to these questions and others must be sought to validate 
the use of hypertext as an instructional system. 
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MENU SELECTIONS 



1. 


Gov«rnm0nt'a Rols 


6. 


Infringemant 
Excaptiona 


2. 


Oafinitions 


7. 


Compansation for 


3. 


Copyrlght/Pat«nt 
Catagoriat 


8. 


Infringament 
Ragistration 


4. 


Eligibility Raquiremants 


9. 


*** Author/Inventor 


5. 


What Conatitutat 
Infringament? 


10. 


Challangaa*** 

Quit 



Click on dctirtd ••(action 



Figure 1 . Tutorial Main Menu 
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